Scheduling In Distributed Systems

In this paper we take a look at three decades of the history and evolution of distributed data processing systems. We consider how allocation of resources to tasks, i.e. scheduling, is done in three systems and study how system goals influence scheduler design and possible scheduling optimizations. We start by examining the first general-purpose distributed computing system Condor [1], continue with the pioneering parallel relational database Gamma [2], and finally introduce contemporary distributed system Map-Reduce [3]. The concepts behind each of the systems are representative for general computing platforms, parallel databases, and Map-Reduce platforms correspondingly.